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Executive  Summary 


The  most  significant  outcome  from  this  workshop  was  the  surfacing  of  a  need  for  better 
understanding  of  how  the  Army  Campaign  Plan  for  the  Future  Force  vision  relates  to 
expected  approaches  for  developing  an  acquisition  strategy  to  achieve  that  vision.  Several 
aspects  of  this  relationship  were  explored,  and  the  effects  of  conflicting  influences  of 
organizational,  regulatory,  and  statutory  demands  were  examined. 

Specifically,  workshop  participants  identified  how  individual  systems’  acquisition  strategies 
need  to  encompass  the  total  system-of-systems  (SoS)  perspective,  with  key  decisions 
informed  by  cross-system  tradeoffs  and  investment  choices  spanning  multiple  programs  and 
appropriations.  These  principles  were  contrasted  with  the  “stovepiped”  nature  of  the  current 
acquisition  environment,  largely  driven  by  the  appropriations  and  requirements  processes. 
This  workshop  highlighted  the  clash  between  the  regulatory  and  statutory  framework  that 
guides  the  acquisition  process,  and  the  demands  of  an  effective  SoS  acquisition  strategy. 
Resolving  this  conflict  lies  at  the  heart  of  the  challenges  that  confront  the  Army  as  it  moves 
towards  the  Future  Force. 

This  workshop  is  envisioned  as  the  first  of  a  series  of  workshops.  Where  this  workshop  was 
structured  to  identify  the  broadest  possible  set  of  issues — given  the  makeup  of  the 
participants — subsequent  workshops  will  emphasize  either  identifying  new  issues  (from  areas 
not  covered  here),  or  probing  more  deeply  into  a  restricted  set  of  issues. 
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Abstract 


This  report  documents  the  proceedings  of  the  Future  Force  Workshop  held  at  the  Software 
Engineering  Institute  on  October  13-14,  2004.  It  describes  the  background  and  motivation  for 
the  workshop,  provides  a  brief  overview  of  the  workshop  activities,  and  highlights  the  key 
observations  and  conclusions  obtained  through  the  course  of  the  workshop  and  post¬ 
workshop  analyses.  In  addition,  a  set  of  recommended  next  steps  is  described. 
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1  Introduction 


1.1  Background  and  Motivation 

The  Army  is  undergoing  a  fundamental  transformation  in  the  organization  of  its  combat  and 
support  forces.  The  transformation  requires  processes  to  acquire  and  sustain  complex, 
interoperable  systems  of  systems  (SoS).  The  goals  for  the  Army’s  transformation  are  outlined 
in  the  Army  Campaign  Plan  (ACP),  which  describes  the  goals  for  the  “Future  Force.”  With 
this  background,  the  Future  Force  Workshop  (FFW)  was  devised  to  initiate  and  facilitate 
discussion  among  key  Future  Force  stakeholders  (with  participation  limited,  initially, 
primarily  to  acquisition  and  headquarters  personnel)  on  the  challenges  to  achieving  and 
sustaining  interoperable  SoS.  Specifically,  this  first  workshop  was  designed  to 

•  help  the  Army  identify  issues,  dependencies,  incompatibilities  and  risks  associated 
with  the  integration  of  systems  in  the  context  of  an  interoperable  Future  Force 

•  explore  use  of  a  workshop  as  a  mechanism  for  eliciting  these  issues 

In  addition,  the  workshop  was  intended  to  help  refine  the  Software  Engineering  Institute’s 
(SEI)  understanding  of  how  the  relationships  between  different  “aspects”  of 
interoperability — programmatic,  constructive,  and  operational — affect  the  ability  to  acquire, 
integrate,  and  field  sustainable,  interoperable  SoS. 


1.2  Report  Structure 

Chapter  2  provides  an  overview  of  the  workshop  including  structure,  participants,  and  a 
discussion  of  how  the  top  issues  were  identified. 

Chapters  3  and  4  presents  details  of  the  significant  findings  from  the  workshop — including 
the  top  issues  and  their  decomposition  into  different  aspects — and  the  results  of  the  post¬ 
workshop  data  analysis. 

Chapter  5  presents  some  conclusions  about  the  relationship  between  the  present  acquisition 
environment  and  the  demands  of  SoS  development,  acquisition,  fielding,  and  sustainment. 
Finally,  some  recommendations  for  future  work  are  presented. 

The  appendices  list  the  organizations  which  participated  in  the  workshop  (Appendix  A),  the 
complete  set  of  interoperability  issues  identified  by  the  workshop  participants  (  Appendix  B), 
and  a  list  of  acronyms  (Appendix  C). 
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Throughout  this  report,  the  authors  attempted  to  present  the  information  exactly  as  it  was 
recorded  during  the  workshop,  in  order  to  ensure  that  we  did  not  change  the  meaning  of  the 
findings.  Furthermore,  in  the  absence  of  any  contemporaneous  clarifications  or  additional 
context,  we  did  not  attempt  to  adduce,  after-the-fact,  supplementary  meaning  to  the  recorded 
statements.  As  a  result,  many  of  the  statements  appear  somewhat  cryptic. 
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2  Workshop  Overview 


2.1  Interoperability  Defined 

Some  definitions  are  required  before  the  results  from  the  workshop  are  described. 
Interoperability  has  traditionally  been  defined  in  an  operational  context  (e.g.,  the  ability  of 
systems  to  exchange  information).  This  definition  is  too  imprecise  and  incomplete  to  describe 
the  essential  characteristics  of  interoperability,  much  less  to  allow  one  to  reason  about 
possible  strategies  to  achieve — and  maintain — interoperability.  In  the  SETs  “System-of- 
Systems  Interoperability”  (SOSI)  report,  Morris  and  associates  discuss  how  interoperability 
is  not  a  property  of  a  system  in  isolation,  but  is  dependent  on  a  particular  context  [Morris 
04].  Specifically,  they  define  interoperability  as 

The  ability  of  a  set  of  communicating  entities  to  (1)  exchange  specified  state 
data  and  (2)  operate  on  that  state  data  according  to  specified,  agreed-upon, 
operational  semantics. 

While  this  definition  addresses  the  issue  of  context,  it  doesn’t  go  far  enough.  The  SOSI  report 
further  identifies  three  distinct — but  interrelated — aspects  of  interoperability  that,  taken 
together  as  a  whole,  provide  a  richer  understanding  of  what  is  meant  by  interoperability. 
Figure  1,  the  SOSI  model,  illustrates  these  aspects  and  the  relationships  between  the 
programmatic,  constructive,  and  operational  aspects  of  interoperability  within  and  across 
programs  [Morris  04,  Meyers  05]. 


Program-1  Program-2 


Figure  1:  The  SOSI  Model 
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The  three  interoperability  aspects  are  characterized  as  follows: 

•  Operational  interoperability  is  closely  aligned  with  the  traditional  definition  of 
interoperability:  the  ability  of  systems  to  exchange  information,  plus  the  additional 
notion  of  compatible  (or  complementary)  operational  concepts. 

•  Constructive  interoperability  reflects  the  degree  to  which  the  different  system  design, 
engineering,  and  production  processes  and  tools  are  able  to  exchange  information  in  an 
understandable  manner. 

•  Programmatic  interoperability  expresses  the  ability  of  programs  to  accurately 
exchange  information  about  the  management  of  the  programs  involved.  This 
information  can  run  the  gamut  from  budget  and  schedule  information  to  details  on  how 
risks  are  interpreted. 

The  emphasis  on  “aspects”  is  critical:  there  is  no  such  thing  as  a  programmatic,  constructive, 
or  operational  interoperability  issue.  Instead,  there  are  interoperability  issues  that  have 
implications  or  “manifestations”  in  any  or  (in  most  cases)  all  three  interoperability  aspects. 
Whereas  traditional  treatments  of  interoperability  largely  ignore  the  constructive  and 
programmatic  aspects,  the  participants  in  the  SOSI  study  concluded  that  their  impact  on 
interoperability  is  significant.  In  fact,  they  concluded  that  the  programmatic  interoperability 
aspects  often  overwhelm  the  operational  and  constructive  aspects.  For  this  reason,  this 
workshop  focused  on  eliciting  interoperability  issues  that  “bear  on”  programmatic 
interoperability,  and  on  understanding  the  interrelationships  between  the  programmatic, 
constructive,  and  operational  interoperability  aspects  of  these  issues. 

For  the  purposes  of  this  workshop  (and  the  remainder  of  this  report),  the  SOSI  model 
definitions  for  interoperability — and  the  different  aspects  of  interoperability — were  used. 


2.2  Workshop  Organization 

The  workshop  took  place  over  two  days.  The  first  day  started  with  some  “stage  setting”  and 
small-group  exercises  to  highlight  the  shortcomings  of  a  (conventional)  program-centric 
approach  to  decision-making.  The  SOSI  model  was  then  presented,  and  a  short  exercise  was 
conducted  to  familiarize  the  participants  with  the  three  aspects  of  interoperability  identified 
by  the  SOSI  model  (programmatic,  constructive,  and  operational).  The  balance  of  the  first 
day  was  spent  brainstorming  issues  related  to  the  Future  Force  transformation.  The  workshop 
participants  were  instructed  to  be  prepared  to  discuss  their  individual  top  two  SoS 
interoperability  issues  the  following  day. 

The  second  day  began  with  recording  the  “top  two”  issues  provided  by  each  workshop 
participant.  As  a  prelude  to  a  brainstorming  session  on  these  issues,  and  to  help  stimulate 
some  ideas  and  discussion,  the  results  of  the  SOSI  IRAD  project  were  briefly  presented.  The 
issues  were  then  prioritized  by  the  participants,  and  the  workshop  participants  were  led 
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through  a  brainstorming  session  to  decompose  the  top  four  issues  into  their  constituent 
aspects  (i.e.,  programmatic,  constructive,  and  operational). 


2.3  Participants 

The  workshop  participants  came  from  various  Army  organizations,  and  reflected  significant 
breadth  and  depth  of  acquisition  experience;  additionally,  some  participants  had  operational 
experience.  Participants  included  representatives  from  Army  headquarters,  staff  elements, 
Joint  and  Army  acquisition  organizations,  operational  test,  research  and  development,  and 
training  and  doctrine.  The  workshop  was  conducted  by  personnel  from  the  SEI  Integration  of 
Software-Intensive  Systems  (ISIS)  initiative  and  the  Acquisition  Support  Program  (ASP).  A 
complete  list  of  participating  organizations  is  provided  in  Appendix  A. 


2.4  Issues  Identification 

After  the  small  group  and  SOSI  exercises  on  the  first  day,  there  was  a  facilitated 
brainstorming  session  to  identify  some  key  barriers  to  achieving  SoS  interoperability,  and  an 
attempt  to  categorize  them  into  one  of  four  types  of  issues:  general,  programmatic, 
constructive,  and  operational.  A  broad  set  of  issues  was  identified,  ranging  from  “rewards  are 
wrong”  to  “would  like  an  ORD  (operational  requirements  document)  for  a  system  of 
systems.” 

At  the  conclusion  of  the  first  day,  it  was  apparent  that  some  additional  structure  was  needed 
to  make  the  most  productive  use  of  the  available  time.  Towards  this  end,  all  workshop 
participants  were  asked  to  identify  their  two  most  important  interoperability  issues,  and  to  be 
prepared  to  discuss  them  on  the  second  day.  The  complete  list  of  identified  issues — from  both 
days — is  provided  in  Appendix  B. 

2.5  Prioritization  of  Issues 

Given  the  relatively  large  number  of  issues  generated,  and  the  comparatively  short  time 
available  for  the  workshop,  the  decision  was  made  to  focus  the  decomposition  and 
subsequent  analyses  to  a  subset  of  the  issues.  To  narrow  the  focus  to  those  issues  of  greatest 
significance  to  the  Army’s  Future  Force,  the  workshop  participants  prioritized  the  issues 
using  multi-voting.  The  four  most  significant  issues  are  discussed  in  Section  3. 


2.6  Decomposition  of  Issues 

After  the  issues  were  prioritized,  the  workshop  participants  dissected  the  four  most 
significant  issues  into  their  constituent  programmatic,  constructive,  and  operational 
interoperability  aspects.  The  results  of  this  decomposition  are  provided  in  Section  3. 
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2.7  Post-Workshop  Analysis 

After  the  conclusion  of  the  workshop,  the  data  were  analyzed  to  identify  and  understand  the 
relations  between  the  interoperability  issues  identified  by  the  participants — and  their 
programmatic,  constructive,  and  operational  aspects.  The  results  of  these  analyses  are 
presented  in  Section  4. 


6 


CMU/SEI-2005-TN-042 


3  Workshop  Results 


The  primary  purpose  of  the  workshop — for  the  Army,  certainly — was  to  .  .identify  issues, 
dependencies,  incompatibilities,  and  risks  associated  with  systems  integration  in  the  context 
of  the  Future  Force.”  This  section  will  discuss  the  issues  identified  by  the  workshop 
participants,  and  their  decomposition  into  programmatic,  constructive,  and  operational 
interoperability  aspects. 


3.1  Issues  Identification 

Over  the  two  days  of  the  workshop,  the  participants  identified  roughly  four  dozen  issues 
bearing  on  the  Army’s  goal  of  fielding  an  interoperable  Future  Force;  the  complete  list  is 
provided  in  Appendix  B.  The  participants  grouped  these  issues  into  three  fairly  broad 
categories:  general  issues,  programmatic  issues,  and  operational  issues.  Additionally,  there 
were  a  number  of  uncategorized  issues  (which,  as  it  turns  out,  was  the  largest  group, 
reflecting  some  of  the  difficulties  in  applying  the  SOS1  model  definitions  too  literally).  Not 
surprisingly  (given  the  demographics  of  the  participants),  most  of  these  issues  were  related  to 
the  acquisition  process  and,  in  particular,  to  the  conflicts  between  the  traditional  system 
acquisition  processes  and  the  processes  believed  necessary  to  successfully  acquire,  develop, 
and  field  an  SoS. 


3.2  Prioritization  of  Issues 

As  noted  above,  most  of  the  issues  that  dominated  the  workshop  discussions  reflected  the 
disconnect  between  the  goals,  methods,  and  awards  employed  in  traditional  system 
acquisition  versus  the  demands  of  SoS  acquisition.  This  is  apparent  in  how  the  workshop 
participants  prioritized  the  issues.  The  top  four  issues  (in  decreasing  order  of  significance) 
identified  were 

1 .  The  Army  is  not  organized  to  develop  a  system  of  systems.  There  is  a  lack  of 
understanding  of  requirements,  money  allocation,  interaction,  and  test.  This  is 
a  consequence  of  the  fact  that,  while  the  Army  fields  operational  capability  as 
integrated  units  of  personnel  and  equipment  in  a  defined  structural  relationship, 
systems  are  procured  individually,  in  response  to  separate  operational  requirements, 
appropriations,  etc.  As  a  result,  the  organization  of  the  acquisition  system  does  not 
inherently  encourage  tradeoffs  across  systems  within  an  SoS,  even  though  these 
tradeoffs  are  necessary  to  maximize  operational  effectiveness  of  the  Army  as  a 
whole. 
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2.  The  procurement  versus  development  lifecycle  models  causes  interoperability 
problems  when  functions  are  implemented.  This  issue  arises  when  different 
systems  that  must  be  interoperable  are  procured  separately.  Interoperability  is 
frequently  defined  by  sets  of  standards  and  interfaces,  with  systems  required  to 
implement  these  in  some  common  fashion.  What  can  happen  is  that  the 
organization  responsible  for  procuring  system  “A”  chooses  for  various  reasons  (i.e., 
funding  profile,  fielding  plans)  to  implement  the  required  standards  in  a  different 
order  than  that  chosen  by  the  acquiring  organization  for  system  “B”  (for  equally 
sound  reasons).  This  can  result  in  the  two  systems  being  non-interoperable  until 
both  have  implemented  all  portions  of  the  specified  standards.  Since  the 
procurement  lifecycles  for  both  systems  are  driven  by  their  individual 
requirements  and  funding  lines,  the  result  is  that  interoperability  is  delayed  for 
unacceptably  long  periods  of  time. 

3.  A  migration  plan  must  be  at  the  appropriate  higher  level,  not  based  on  a 
bottom-up  perspective:  Network  not  the  radios,  fielding.  This  issue  reflects  the 
disconnect  between  the  present  approach  to  migration  planning  (i.e.,  system  by 
system)  and  the  need  to  plan  migration  at  the  force  capability  level  (e.g.,  Future 
Force).  Unchecked,  this  issue  can  lead  to  a  decrease  in  interoperability  if — as  is 
frequently  the  case — an  upgraded  system  does  not  provide  an  exact  “form-fit- 
function”  replacement  for  its  predecessor.  One  example  cited  by  the  participants 
involved  a  new  system  being  fielded  to  operational  units  that  were  required  to 
interoperate:  as  each  unit  received  the  new  system — in  accordance  with  the  fielding 
plan  for  the  new  system — the  old  system  was  removed  from  the  unit. 

Unfortunately,  the  new  system  wasn’t  fully  backwards-compatible  with  the  old 
system,  and  the  system  fielding  plan  didn’t  reflect  the  operational  reality  that  these 
units  would  have  to  deploy  and  work  together,  so  until  all  of  the  units  received  the 
new  system,  there  was  a  net  loss  of  interoperability  and  a  resulting  loss  of 
operational  effectiveness. 

4.  There  is  a  need  for  a  process  for  measuring  the  operational  benefit  of  proposed 
interoperability  solutions  (e.g.,  cost-benefit  analysis).  Because  so  much  of  the 
focus  in  justifying  system  upgrades  and  migration  of  new  capability  is  driven  by  the 
individual  systems’  cost-benefits  analysis,  there  is  no  agreed-upon  mechanism  for 
performing  such  analyses  at  the  “system-of-systems”  level.  Or,  as  pointed  out  by 
some  of  the  workshop  participants,  where  such  analyses  are  performed,  they  are 
frequently  driven  by  the  procurement/fielding/sustainment  costs  of  the  proposed 
upgrade,  versus  the  original  system.  This  generally  results  in  the  Analysis  of 
Alternatives  (AOA)  reflecting  a  locally  optimized  solution  for  an  individual 
program  or  system,  rather  than  a  measure  of  the  operational  benefits  of  the 
proposed  upgrades  in  the  larger  perspective. 
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3.3  Detailed  Issues  Decomposition 


3.3.1  Overview  of  the  Decomposition  Process 

The  issues  described  above  were  then  subjected  to  further  brainstorming  to  decompose  them 
into  their  constituent  interoperability  aspects.  That  is,  the  participants  identified  how  each 
issue  was  reflected  in  the  interoperability  aspects  described  by  the  SOSI  model: 
programmatic,  constructive,  and  operational.  For  example,  the  first  issue  relating  to  the 
overall  Army  organization  (“The  Army  is  not  organized  to  build  a  system  of  systems.  There  is 
a  lack  of  understanding  of  requirements,  money  allocation,  interaction,  and  test”)  is  a 
composite  of  the  individual  interoperability  aspects,  as  examined  from  the  programmatic, 
constructive,  and  operational  perspectives.  This  is  illustrated  in  Figure  2. 


•  Use  of  LSI  as  integrator  (vice 
prime) 

•  Requirements 

•  SSEI  not  defined  in  FARS 
(like  SETD) 


•  Schedule  slip;  no 
synchronization  of  schedules 

Incentive 

•  No  alignment  of  delivered 
capability 


•  Disconnect  with 
acquisition  strategies 


Figure  2:  Illustrative  Interoperability  Issue  Decomposition 

3.3.2  Decomposition  Results 

Each  of  the  four  most-significant  issues  (as  identified  and  prioritized  by  the  workshop 
participants)  was  examined  to  identify  its  programmatic,  constructive,  and  operational 
aspects  as  described  above.  The  results,  as  identified  by  the  workshop  attendees,  are  detailed 
in  Tables  1  through  4. 
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Table  1:  Issue  #1  Decomposition 


Issue  # 1 :  The  Army  is  not  organized  to  develop  a  system  of  systems.  There  is  a  lack  of 
understanding  of  requirements,  money  allocation,  interaction,  and  test. 

Programmatic 

Constructive 

Operational 

Schedule  slip;  no 
synchronization  of  schedules 

Use  of  Lead  System 

Integrator  (LSI)  as  integrator 
(vice  prime) 

Who  supports? 

Incentive 

Requirements 

Disconnect  with  acquisition 
strategies 

No  alignment  of  delivered 
capability 

System-of-systems 
Engineering  and  Integration 
(SSEI)  not  defined  in  Federal 
Acquisition  Regulations 
(FARS)  (like  System 
Engineering  and  Technical 
Direction  (SETD)) 

Shift  in  training  focus  from 
“person”  to  “SOS  readiness” 

Cost  overruns 

Specifications  don’t 
collaborate 

No  integrated  approach  to 
get  modularity  (like  Army 
Universal  Task  List  (AUTL)) 

Interoperability  guaranteed  to 
fail 

Contractors  not  structure/ 
incentivized  to  cooperate 

Ineffective  resource 
management 

Capability  Maturity  Model® 
(CMMI®)  level  required 

No  unity  of  effort  between 
acquisition  and  users 

Lack  understanding  of  SOS 
requirements 

Manage  $$,  not  engineering 

Test,  fielding,  ??? 

Vision,  policies,  strategy, 
implementation 

Proprietary  data,  intellectual 
property 

No  ability  to  adjust  budgets 

Inability  to  identify/  resolve 
conflict 

Program  Initiation  Team 
(PIT)  crew 

Capability  Maturity  Model  and  CMM  are  registered  in  the  U.S.  Patent  and  Trademark  Office  by 
Carnegie  Mellon  University. 
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Table  2:  Issue  #2  Decomposition 


Issue  #2:  The  procurement  versus  development  lifecycle  models  cause  interoperability 
problems  when  functions  implemented. 

Programmatic 

Constructive 

Operational 

Need  process  for  developing  an 
SOS  that  emphasizes  software 

No  System  Engineering 
Management  Plan  (SEMP) 

Functions  to  a  block 

Cost/schedule/performance 

Low  price  estimates  lead  to 

cost  overruns 

Functional  capability 

No  repeatable  process  on  the 
development  side 

Inadequate  Gov’t  visibility 
into  contractor  SEMP,  etc., 
means  you  can’t  “do”  an 

SOS 

Software  blocking 

Process  mismatch 

SEMP  for  SOS 

What  to  do? 

Table  3:  Issue  #3  Decomposition 


Issue  #3:  A  migration  plan  must  be  at  the  appropriate  higher  level,  not  based  on  a  bottom- 
up  perspective.  Network  not  the  radios,  fielding. 

Programmatic 

Constructive 

Operational 

Need  a  vision  of  what  you’re 
trying  to  achieve 

What  engineering?  At  what 
levels? 

Need  capability  to  assess 
operational  implications  of 
system  issues 

Who  does  the  migration 
plan? 

Migration  plans  must 
accommodate  technology 
insertion 

Interoperability  can  be 
destroyed  by  making 
“wrong”  decisions. 

There  is  no  migration  plan 
for  the  migration  plans. 

How  to  ensure  consistency 
between  migration  plan  and 
SEMP? 

Lack  of  coordinated 
migration  plans  can  result  in 
delays  in  fielding  capability 

Need  SOS  plan  that 
encompasses  components 

No  SOS  migration  plan 

Use  Clinger-Cohen  Act 
(CCA)  as  a  “wedge”  to  force 
SOS-level  decisions. 

CMU/SEI-2005-TN-042 


11 


Need  an  Army-wide  network 
migration  plan 

Visibility  into  migration 
plans  for  individual  systems 
which  comprise  the  SOS 

Who  has  authority  to 
harmonize  migration  plans? 

SOS  is  held  “hostage”  to 
other  systems  within  the 

SOS 

Not  organized  correctly 

Consider  impacts  of 
individual  system  decisions 
on  entire  SOS 

Who  should  manage  change? 

How  to  make  cross-system 
trades? 

Not  managing  change: 
change  is  managing  us 

What  types  of  activities? 

Bill  to  fix  mistakes  impacts 
future  capability 

Below-Threshold 
Reprogramming  (BTR) 
thresholds  too  low  to  allow 
timely  investment  decisions 


Table  4:  Issue  #4  Decomposition 


Issue  # 4 :  There  is  a  need  for  a  process  for  measuring  operational  benefit  of proposed 
interoperability  solutions  (e.g.,  cost-benefit  analysis). 

Programmatic 

Constructive 

Operational 

AOA  not  effective  at 
assessing  unit’s  ability  to 
perform  mission 

Program  Manager  (PM)  needs 
ability  to  assess  SOS 
operational  interoperability 
implications  of  proposed 
solutions 

Cannot  do  AOA  in 

“vacuum” 

Ability  to  expose/exploit 
existing  interoperability 
solutions  across  SOS 
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Need  to  perform  AOA  for 
the  SOS 


PM/Program  Executive 
Officer  (PEO)  tenure  too  short 


(Lack  of  time  prevented 
identification  of  operational 
aspects  for  this  issue) 


Need  periodic  follow-on 


AOAs 

AOAs  only  consider  “new” 
proposed  solutions — should 
consider  existing  systems, 
too 

AOAs  too  often  simply  a 
“check  in  a  box”  as  opposed 
to  something  useful 

AOAs  should  be  mission- 
funded  (  vice  by  a  PM  with  a 
vested  interest  in  the 
outcome) 


Note:  It  is  possible  to  argue  over  the  precise  classification  of  specific  elements  into 
programmatic,  constructive,  or  operational  interoperability  aspects.  The  above  tables  reflect 
the  decisions  of  the  workshop  participants.  To  the  extent  that  there  is  debate  about  the 
appropriateness  of  one  placement  versus  another  for  a  given  element  reflects  on  the  relative 
inexperience  of  the  participants  with  the  SOSI  model,  as  well  as  the  difficulty  in  “cleanly” 
separating  complex  issues  into  their  constituent  components. 

To  put  this  data  into  an  understandable  context,  and  allow  meaningful  conclusions  to  be 
drawn,  the  workshop  data  was  subjected  to  a  variety  of  affinity  grouping  and  graphing 
techniques  during  post-workshop  data  analysis.  This  process — and  results — are  described  in 
more  detail  in  the  next  section. 
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4  Post-Workshop  Data  Analysis 


Post-workshop  data  analysis,  performed  by  the  SEI,  focused  on  identifying  and 
understanding  the  key  relationships  between  the  programmatic,  constructive,  and  operational 
aspects  of  the  top  four  issues  identified  by  the  workshop  participants.  The  purpose  for  this 
analysis  was  twofold: 

1 .  to  obtain  some  insights  into  what  the  data — the  issues  and  their  respective  aspects 
identified  by  the  workshop  participants — revealed  as  the  root  causes  for  these  issues 

2.  to  develop  a  framework  for  assessing  possible  solutions. 

The  following  sections  describe  how  the  post-workshop  analysis  was  accomplished  and 
highlights  some  of  the  immediate  results  from  these  analyses. 


4.1  Intra-Issue  Affinity  Graphs 

The  first  step  in  the  post-workshop  analysis  was  to  examine  each  of  the  top  four  issues  in 
isolation,  to  see  what  cross-aspect  affinity  relationships  existed  between  its  constituent 
components.  These  were  represented  in  a  series  of  undirected  graphs,  where  the  nodes 
represent  the  decomposition  of  the  issues  into  their  respective  interoperability  aspects,  and 
the  arcs  indicate  a  relationship.  These  are  shown  in  Figures  3,  4,  5,  and  6. 


The  graph  symbology  is  explained  below: 


Schedule  slip; 
no 

synchronization 
of  schedules 
6 


In  Figure  3,  node  6  is  labeled  “Schedule  slip;  no  synchronization  of 
schedules.”  This  node  is  connected  to  seven  other  nodes  by  a  set  of  arcs.  The 
node  colors  and  cross-hatching  indicate  the  particular  interoperability  aspect: 


yellow  for  programmatic,  pink  for  constructive,  and  orange;  for  operational. 


Disconnect  with 
acquisition 
strategies 
2 


Schedule  slip; 

no 

synchronization 
of  schedules 
6 


The  arcs  between  node  6  and  these  other  nodes  indicate  an  apparent 
relationship  between  these  specific  aspects  of  the  interoperability  issue.  For 
example,  the  fact  that  there  is  a  disconnect  with  acquisition  strategies  (node  2) 
seems  to  contribute  to  schedule  slips. 

Note  that  the  arcs  are  non-directional:  the  emphasis  here  is  on  the  existence  of 
relationships,  not  their  causal  or  semantic  interpretations.  Additionally,  arcs 


indicate  relations  between  nodes  in  different  aspects:  since  there  is  a  strong  correlation 


between  the  nodes  within  a  given  aspect,  it  is  believed  that  the  cross-aspect  relations  provide 


greater  insight  into  the  underlying  interoperability  issue. 


14 


CMU/SEI-2005-TN-042 


Shift  in 
training 
focus  from 
“person"  to 
“SOS 
readiness” 
15 


No 

integrated 
approach 
to  get 
modularity 


Disconnect 

with 

acquisition 

strategies 


Who 

supports? 


Schedule  slip;  no 
synchronization 
of  schedules 


No  alignment  of 
delivered 
capability 


SSEI  not  defined  in 
FARs  (like  SETD) 
22 


Incentive 

24 


Manage  $$,  not 
engineering 


_ _ 


Use  of  LSI  as 
integrator  (vice 
prime) 


Interoperability 
guaranteed  to  fail 


Proprietary  data, 
intellectual 
property 


No  unity  of  effort 
between  acquisition 
and  users 


Ineffective 

resource 

management 


Requirements 


collaborate 

18 


Inability  to 
identify/ 
resolve 
conflicts 


Program 
Initiation 
Team  (PIT) 
crew 


Vision,  policies, 
strategy, 
implementation 


No  ability 
to  adjust 
budgets 
13 


of  SOS 
requirements 


tors  m 

* 

• 

r 

j 

CMM  level 
required 

23 

_ ? 

Figure  3:  Issue  #1:  The  Army  is  not  organized  to  develop  a  system  of  systems. 

There  is  a  lack  of  understanding  of  requirements,  money  allocation, 
interaction,  and  test. 
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Figure  4:  Issue  #2:  The  procurement  versus  development  lifecycle  models  causes 
interoperability  problems  for  when  functions  implemented. 
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There  is  no 
migration  plan  for 
the  migration 
plans 


Need  an  Army¬ 
wide  network 
migration  plan 


Lack  of  coordinated 
migration  plans  can 
result  in  delays  in 
fielding  capability 


Need  a  vision  of 
what  you’re  trying 
to  achieve 


Bill  to  fix  mistakes 
impacts  future 
capability 


What  types  of 
activities? 
12 


Visibility  into 
migration  plans  for 
individual  systems 
which  comprise  the 
SOS 


SOS  is  held 
“hostage”  to  other 
systems  within  the 
SOS 
15 


Need  SOS  plan 
that 

encompasses 

components 


How  to  make 
cross-system 
trades? 


Consider  impacts  of 
individual  system 
decisions  on  entire 
SOS 


No  SOS  migration 
plan 
17 


Not  managing 
change:  change 
is  managing  us 


How  to  ensure 
consistency 
between 
migration  plan 
and  SEMP? 
20 


Migration  plans  must 
accommodate 
technology  insertion 


What 

engineering?  At 
what  levels? 
19 


Who  should 
manage  change? 

3 

Who  has  authority 
to  harmonize 
migration  plans? 

7 

Who  does  the 
migration  plan? 

6 

Not  organized 
correctly 

Need  capability  to 

Interoperability  can 

assess  operational 

be  destroyed  by 

implications  of 

making  "wrong” 

system  issues 

decisions 

21 

22 

Use  CCA  as  a  “wedge” 
to  force  SOS-level 
decisions 

24 

BTR  thresholds 
too  low  to  allow 
timely 
investment 
decisions 
13 


Figure  5:  Issue  #3:  A  migration  plan  must  be  at  the  appropriate  higher  level,  not 
based  on  a  bottom-up  perspective.  Network  not  the  radios,  fielding. 1 


1  Figure  5  is  shown  in  a  “collapsed”  form,  where  related  items  within  each  of  the  three  interoperability 
aspects  are  grouped  together.  Using  the  same  format  as  the  other  graphs  resulted  in  more  than  130 
arcs,  rendering  the  graph  nearly  indecipherable. 
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AOA  not  effective  at 
assessing  unit's 
ability  to  perform 
mission 


PM/PEO  tenure 
too  short 


AOAs  should  be 
mission-funded  (vice  by 
a  PM  with  a  vested 
interested  in  the 
outcome) 


PM  needs  ability  to  assess 
SOS  operational 
interoperability  implications 
of  proposed  solutions 


Cannot  do  an 
AOA  in 
“vacuum" 


Ability  to  expose/exploit 
existing  interoperability 
solutions  across  SOS 


AOAs  only  consider  "new" 
proposed  solutions— should 
consider  existing  systems, 
too 


AOA's  too  often  simply  a 
“check  in  a  box"  as 
opposed  to  something 
useful 


Need  to 
perform  AOA 
for  SOS 


Need  periodic 
follow-on 
AOAs 


Figure  6:  Issue  #4:  There  is  a  need  for  a  process  for  measuring  operational  benefit 
of  proposed  interoperability  solutions  (e.g.,  cost-benefit  analysis). 2 


While  this  process  revealed  the  presence  of  significant  coupling  between  the  different 
interoperability  aspects  of  the  top  four  issues,  it  didn’t  afford  any  particular  insights  into  the 
larger  issues  of  acquiring,  fielding,  and  sustaining  an  interoperable  Future  Force.  This 
indicated  the  need  for  additional  analysis  to  attempt  to  identify  patterns  in  relationships 
across  the  top  four  issues. 


2  As  noted  in  Table  4,  lack  of  time  precluded  identification  of  any  operational  interoperability  aspects 
for  this  issue. 
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4.2  Cross-Issue  Affinity  Groupings 

Following  the  intra-issue  analysis,  the  collected  issue  aspects  were  examined  to  discover  any 
affinity  groupings — within  each  aspect — spanning  the  top  four  issues.  In  other  words,  were 
there  apparent  affinity  groupings  within  the  programmatic,  constructive,  and  operational 
aspects  across  the  four  issues?  The  following  tables  represent  these  affinity  groups.  The  table 
title  indicates  the  name  given  to  the  affinity  group;  the  first  column  lists  the  top  four  issues, 
and  the  second  column  indicates  which  aspects — from  each  issue — were  collected  into  the 
affinity  group. 


Programmatic  SoS  Leadership  Issues  (p_sos_leadership) 


Issue 

Programmatic  SoS  Leadership  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

Vision,  policies,  strategy,  implementation 

Interoperability  guaranteed  to  fail 

Program  Initiation  Team  (PIT)  crew 

#2  -  Procurement  versus  development 
lifecycle  models 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

Need  an  Army-wide  network  migration  plan 

Who  should  manage  change? 

Not  organized  correctly 

Use  CCA  as  a  “wedge”  to  force  SoS-level 
decisions 

#4  -  Need  process  for  measuring  operational 
benefit 

Acquisition  Strategy  Issues  (acq_strategy) 


Issue 

Acquisition  Strategy  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

#2  -  Procurement  versus  development 
lifecycle  models 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

Need  a  vision  of  what  you’re  trying  to 
achieve 
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Interoperability  can  be  destroyed  by  making 
“wrong”  decisions 

#4  -  Need  process  for  measuring  operational 
benefit 

SoS  Management  Issues  (sos_mgmt) 


Issue 

SoS  Management  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

Incentives 

Manage  $$,  not  engineering 

No  unity  of  effort  between  acquisition  and 

users 

Inability  to  identify/resolve  conflicts 

No  ability  to  adjust  budgets 

There  is  no  migration  plan  for  migration 
plans 

Need  SoS  plan  that  encompasses  components 

Who  has  authority  to  harmonize  migration 
plans? 

#2  -  Procurement  versus  development 

Need  process  for  developing  an  SoS  that 

lifecycle  models 

encompasses  software 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

Who  does  the  migration  plan? 

#4  -  Need  process  for  measuring  operational 
benefit 

PM/PEO  tenure  too  short 
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SoS  Processes  Issues  (sos_process) 


Issue 

SoS  Processes  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

#2  -  Procurement  versus  development 
lifecycle  models 

Process  mismatch 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

#4  -  Need  process  for  measuring  operational 
benefit 

SoS  Analysis  of  Alternatives  Issues  (sos_aoa) 


Issue 

SoS  Analysis  of  Alternatives  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

#2  -  Procurement  versus  development 
lifecycle  models 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

#4  -  Need  process  for  measuring  operational 
benefit 

AO  As  too  often  simply  a  “check  in  the  box” 
as  opposed  to  something  useful 

Can’t  do  AOA  in  a  vacuum 

Need  periodic  follow-on  AOAs 

AOAs  not  effective  at  assessing  unit’s  ability 
to  perform  mission 

Need  to  perform  AOA  for  the  SoS 

AOAs  only  consider  “new”  proposed 
solutions — should  consider  existing  systems, 
too 

AOAs  should  be  mission-funded  (vice  by  a 

PM  with  a  vested  interest  in  the  outcome) 
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SoS  Execution  Issues  (sos_execution) 


Issue 

SoS  Execution  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

Schedule  slip;  no  synchronization  of  schedules 

No  alignment  of  delivered  capabilities 

Ineffective  resource  management 

Cost  overruns 

#2  -  Procurement  versus  development 
lifecycle  models 

Cost/schedule/performance 

No  repeatable  process  on  the  development  side 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

Bill  to  fix  mistakes  impacts  future  capability 

How  to  make  cross-system  trades? 

#4  -  Need  process  for  measuring 
operational  benefit 

System-of-System  Engineering  Issues  (sos_engineering) 


Issue 

SoS  Engineering  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

Individual  system  requirements  clash 

Specifications  don’t  collaborate 

#2  -  Procurement  versus  development 
lifecycle  models 

Inadequate  Gov’t  visibility  into  contractor 
SEMP,  etc.,  means  you  can’t  “do”  an  SoS 

SEMP  for  SoS 

No  SEMP 

Lack  of  understanding  of  SoS  requirements 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

What  engineering?  At  what  levels? 

SOS  is  held  “hostage”  to  other  systems 
within  the  SOS 

Consider  impacts  of  individual  system 
decisions  on  entire  SOS 

#4  -  Need  process  for  measuring  operational 
benefit 

PM  needs  ability  to  assess  SOS  operational 
interoperability  implications  of  proposed 
solutions 
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Contractor  Management  Issues  (contractor_mgmt) 


Issue 

Contractor  Management  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

Contractors  no  structure/incentivized  to 

cooperate 

CMM  level  required 

Use  of  LSI  as  integrator  (vice  prime) 

SSEI  not  defined  in  FARS  (like  SETD) 

Proprietary  data,  intellectual  property 

#2  -  Procurement  versus  development 
lifecycle  models 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

#4  -  Need  process  for  measuring  operational 
benefit 

Constructive  SoS  Leadership  Issues  (c_sos_leadership) 


Issue 

Constructive  SoS  Leadership  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

#2  -  Procurement  versus  development 
lifecycle  models 

What  to  do? 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

Who  has  authority  to  harmonize  migration 
plans? 

What  types  of  activities? 

#4  -  Need  process  for  measuring  operational 
benefit 
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Cost  Estimating  Issues  (cost_estimate) 


Issue 

Cost  Estimating  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

#2  -  Procurement  versus  development 
lifecycle  models 

Low  price  estimates  lead  to  cost  overruns 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

#4  -  Need  process  for  measuring  operational 
benefit 

Reuse  Issues  (reuse) 


Issue 

Reuse  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

No  integrated  approach  to  get  modularity 
(like  AUTL) 

#2  -  Procurement  versus  development 
lifecycle  models 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

Need  SoS  plan  that  encompasses  components 

#4  -  Need  process  for  measuring  operational 
benefit 

Ability  to  expose/exploit  existing 
interoperability  solutions  across  SoS 

Constructive  Evolution  Issues  (c_evolution) 


Issue 

Constructive  Evolution  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

#2  -  Procurement  versus  development 
lifecycle  models 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

Consistency  between  migration  plans  and 
SEMP 

Migration  plans  must  accommodate  tech 
insertion 

No  SoS-level  migration  plan 
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Visibility  into  migration  plans  for  individual 
systems  which  comprise  the  SoS 

#4  -  Need  process  for  measuring  operational 
benefit 

Force  Structure  Issues  (force_structure) 


Issue 

Force  Structure  Aspect 

#1  -  Not  organized  to  build  a  system  of 

Who  supports? 

systems 

Disconnect  with  acquisition  strategies 

#2  -  Procurement  versus  development 

Functional  blocking 

lifecycle  models 

Software  blocking 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

#4  -  Need  process  for  measuring  operational 
benefit 

Field  Operational  Capability  Issues  (field_ops_capability) 


Issue 

Field  Operational  Capability  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

Shift  in  training  focus  from  “person”  to  “SoS 
readiness” 

#2  -  Procurement  versus  development 
lifecycle  models 

Functions  to  a  block 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

#4  -  Need  process  for  measuring  operational 
benefit 
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Assess  Capabilities  Issues  (assess_capabilities) 


Issue 

Assess  Capabilities  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

#2  -  Procurement  versus  development 
lifecycle  models 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

Need  capability  to  assess  operational 
implications  of  system  issues 

#4  -  Need  process  for  measuring  operational 
benefit 

Operational  Evolution  Issues  (o_evolution) 


Issue 

Operational  Evolution  Aspect 

#1  -  Not  organized  to  build  a  system  of 
systems 

#2  -  Procurement  versus  development 
lifecycle  models 

#3  -  Migration  plan  must  be  at  the 
appropriate  higher  level 

Not  managing  change:  change  is  managing 

us 

Lack  of  coordinated  migration  plans  can 
result  in  delays  in  fielding  capability 

#4  -  Need  process  for  measuring  operational 
benefit 

Similar  to  the  previous  analysis  stage,  this  process  revealed  some  additional  insights  into  the 
relationships  between  the  various  interoperability  aspects  across  the  top  four  issues,  but  failed 
to  bring  out  the  higher  level  programmatic  issues.  However,  further  reflection  leads  to  the 
realization  that  these  tables  can  be  organized  into  three  broad  categories:  programmatic, 
constructive,  and  organizational.  These  cross-issue  intra-aspect  groupings  and  their  natural 
division  into  programmatic,  constructive,  and  operational  aspects  suggest  a  final  analysis 
step. 
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4.3  Interrelations  Among  Cross-Aspect  Affinity  Groups 

A  review  of  the  cross-issue  groupings  and  contemporaneous  notes  from  the  workshop 
showed  relationships  between  groupings  within  each  of  the  three  interoperability  aspects,  as 
well  as  relationships  between  groupings  cutting  across  the  aspects.  Furthermore,  there 
appeared  to  be  different  degrees  of  coupling  (“stronger”  or  “weaker”),  as  well  as  some  sense 
of  possible  causality.  The  cross-cutting  relationships — representing  “touch  points”  between 
the  programmatic,  constructive,  and  operational  aspects — were  identified  (“cross-aspect” 
relationships),  and  a  “critical  chain”  of  relationships  was  identified.  These  are  shown  in 
Figure  7. 


This  diagram  (Figure  7)  requires  some  explanation: 


The  solid  arrow  from  psosleadership  to  acqstrategy  indicates  a 
stronger  coupling  from  the  former  to  the  latter.  This  implies  that  the 
activities  and  responsibilities  attendant  in  acq  strategy  are  in  response  to, 
among  other  sources,  direction  from  activities  resident  in 
p  sos  leadership.  The  solid  arrow  denotes  the  strong  relation,  as  well  as 
its  causal  nature.  (Note  that  the  relation  is  not  strictly  one-way;  the  arrow  indicates  the 
“dominant”  direction  of  the  relation.) 


The  thicker  green  arrow  into  acq  strategy  represents  a  cross-cutting 
relation  from  a  group  in  another  aspect  (in  this  case,  operational 
interoperability).  Similarly,  the  bi-directional  green  arrow  between 
sosmgmt  and  contractor  mgmt  indicates  a  dual  relation  between  these 
groupings  that  spans  the  boundary  separating  programmatic  and  constructive  interoperability. 
The  dual  relation  implies  that  the  activities  and  responsibilities  in  both  groups  are  in  response 
to,  or  informed  by,  the  other. 

C  j  The  dashed  line  from  sos  aoa  and  sos  execution  indicates  a  weaker 

coupling  between  these  two  groups  (that  is,  weaker  than  the  degree  of 
coupling  indicated  by  a  solid  line). 


acq  strategy ,  sos  mgmt,  and  contractor  mgmt  are  decorated  with  red 
ellipses:  these  indicate  that  these  groups  are  part  of  the  critical  chain  of 
relations  that  are  necessary  for  success  in  fielding  the  Future  Force. 


A  key  point  to  remember  when  looking  at  these  diagrams  is  that  they  represent  what  the 
workshop  participants  described  as  necessary  for  a  successful  SoS  implementation.  Flow  they 
described  this  was  largely  through  enumerating  the  shortfalls  with  the  current  processes  and 
acquisition  framework  (e.g.,  regulations,  laws,  etc.).  Their  issues  defined  the  problems; 
conversely,  things  outside  of  the  issues  represent  what  is  needed.  The  significance  of  this  will 
be  explained  in  the  following  sections. 
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Figure  7;  Cross-Aspect  Interoperability  Relationships 
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:  Critical  chain 


4.4  Integration  of  Post-Workshop  Analyses 

It  was  only  after  completing  all  of  the  aforementioned  analyses  that  an  articulable  “theme” 
began  to  emerge.  This  theme,  which  underlies  many  of  the  conclusions,  was  the  necessity  of 
a  strong  cross-aspect  coupling  between  key  affinity  groups  (the  so-called  “touch  points” 
mentioned  earlier).  This  is  shown  in  Figure  7  by  the  solid  green  arrows.  In  looking  at  this 
graph,  a  few  points  stand  out: 

1 .  A  need  exists  for  a  linkage  between  the  desired  force  structure  (and  attendant  operational 
concepts)  and  the  associated  acquisition  strategy  required  to  bring  this  into  being.  The 
existence  of  this  touch  point  between  the  operational  and  programmatic  interoperability 
aspects  means  that  the  processes,  concerns,  issues,  and  so  forth,  in  one  area  (i.e., 

“force  structure”)  must  be  informed  by,  and  compatible  with,  their  corresponding 
equivalents  in  the  other  area  (i.e.,  “acq_strategy”). 

2.  Contractor  management — traditionally  viewed  as  more  of  a  programmatic  concern — is 
actually  both  a  programmatic  and  a  constructive  interoperability  concern.  The  structure 
of  the  contractual  relationships  between  the  acquirer(s)  and  developer(s)  in  an  SoS 
context  influences  every  aspect  of  the  eventual  SoS.  For  example,  if  the  SoS  requires 
close  collaboration  between  developers  of  different  components  for  the  success  of  the 
SoS,  inappropriate  contractual  language  could  actually  preclude  the  sharing  of  critical 
intellectual  property  between  developers  and  lead  to  the  failure  of  the  SoS  (or  at  least  a 
significant  loss  in  anticipated  capability,  operational  utility,  etc.). 

3.  The  touch  points  between  the  programmatic  analysis  of  alternatives  (“sos  aoa”)  and  the 
assessment  of  operational  capabilities  of  the  SoS  (“assess  capabilities”)  indicates  that 
individual  programs’  analysis  of  alternatives  need  to  be  done  in  the  context  of  the  SoS. 
The  “optimum”  choice  for  a  system  in  isolation  will  most  likely  not  be  the  best — from 
an  SoS  perspective — when  that  system  is  placed  into  the  broader  context. 

4.  Similarly,  the  constructive  and  operational  aspects  of  system  evolution  (indicated  by  the 
“evolution”  groups  under  the  constructive  and  operational  interoperability  aspects)  need 
to  be  consistent:  evolution  of  a  system  must  be  undertaken  in  the  context  of  the  evolving 
SoS. 

The  next  section  will  highlight  the  conclusions  drawn  from  the  results  of  the  workshop  and 
post-workshop  data  analysis,  and  will  make  some  recommendations  about  possible  “next 
steps.” 
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5  Conclusions 


The  workshop  participants  identified  four  key  issues  related  to  the  acquisition  of  the  Army 
Future  Force.  The  issues  related  to 

1.  the  need  for  a  required  organizational  approach  to  an  SoS  acquisition 

2.  disconnects  between  the  development  and  procurement  lifecycle  models 

3.  the  need  to  address  migration  plans  in  the  context  of  an  SoS 

4.  understanding  operational  benefits  at  the  SoS  level 


All  of  these  issues  stress  the  contrast  between  the  perspectives  of  an  individual  program, 
acting  with  a  fairly  high  degree  of  autonomy,  versus  that  of  a  PMO  executing  in  the  context 
of  a  system-of-systems.  Two  fundamental  mechanisms  give  rise  to  these  issues: 

1 .  an  asymmetry  between  the  operational  view  (in  terms  of  a  system  of  systems)  and  the 
program-centric  view  (of  a  specific  system);  reinforced  by 

2.  influences  of  the  current  acquisition  environment 

5.1  Operational  View  Versus  Program-Centric  View 

The  workshop  participants  repeatedly  highlighted  issues  resulting  from  a  disconnect  between 
the  way  capabilities  are  implemented  operationally,  as  collections  of  systems  of  systems,  and 
the  manner  in  which  they  are  procured,  developed,  and  fielded:  as  individual  systems.  This 
results  in 

1.  issues  related  to  the  development,  procurement,  and  fielding  of  a  software-intensive 
system  (represented  by  a  vertical  “slice”  through  a  single  program  in  the  SOSI  model). 
This  inward  focus,  or  “PMO-centric”  view,  reflects  the  traditional  program  management 
view  of  “their”  system  as  the  center  of  the  universe. 

2.  issues  arising  from  the  disconnect  between  the  goals,  methods,  and  rewards  that  have 
been  developed  for  traditional  (i.e.,  “stovepiped”)  system  procurement  and  the  realities 
of  an  SoS  approach  (represented  in  the  SOSI  model  by  the  horizontal  linkages  between 
individual  programs).  This  reflects  an  “SoS-centric”  perspective,  where  systems  exist  in 
the  context  of  an  SoS. 

An  example  of  issues  that  fall  into  the  first  category  includes  observations  like  “how  you  buy 
a  truck  is  different  from  how  you  deliver  software”  and  “proliferation  of  requirements.” 
Examples  of  the  second  category  include  “acquisition  process  is  not  defined  for  an  SoS”  and 
“rewards  are  wrong.”  Examining  these  two  broad  categories,  in  turn,  reveals  additional 
groupings  that  reflect  the  SOSI  model’s  programmatic,  constructive,  and  operational  aspects. 
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5.2  Influences  of  the  Current  Acquisition  Environment 

A  recurring  theme  from  the  workshop  is  the  perception  that  the  lack  of  flexibility  in  the 
existing  regulatory  and  statutory  framework  makes  it  impossible  to  “do  the  right  thing”  for  an 
SoS  acquisition.  Specifically,  the  existing  framework  is  strongly  oriented  towards  a  program¬ 
centric  view  (e.g.,  funds  are  appropriated  for  specific  programs,  program  execution  is  at  the 
individual  program  level,  etc.)  versus  an  SoS  view.  This  orientation  exists  for  a  number  of 
reasons,  not  the  least  of  which  is  the  fact  that  system  development,  acquisition,  and  fielding 
has  “always”  been  done  that  way.  This  has  resulted  in  the  creation  of  a  “safe  harbor” 
mentality  within  the  acquisition  community:  program  managers  (and  developers,  etc.)  have 
always  been  able  to  fall  back  on  the  “satisfying  the  ORD  (or  contract  specifications,  etc.)” 
defense.  While  it  used  to  be  true  that  “nobody  ever  got  fired  for  following  DoD  5000.2,”  the 
emphasis  on  SoS  means  that  the  acquisition  community  is  finding  it  increasingly  difficult  to 
seek  refuge  in  traditional  definitions  of  success.  In  other  words,  doing  what  you’ve  always 
done  isn’t  “good  enough”  for  a  system-of-systems. 


5.3  Next  Steps 

The  preceding  suggests  that  a  larger  perspective  is  needed.  An  overarching  principle  emerges: 

Program  management  organizations  need  to  execute  in  a  manner  that  is  consistent  with 
the  larger  system-of-systems  view.  Achieving  this  goal  requires 

•  a  vision,  not  just  of  the  SoS,  but  of  how  programs  must  work  together  to  achieve  that 
vision,  including  an  explicit  linkage  between  the  operational  architecture  (as  reflected  by 
evolving  force  concepts,  doctrine,  etc.)  and  the  acquisition,  development,  and  fielding  of 
programs  within  the  SoS 

•  systems  engineering  at  the  SoS  level — with  corresponding  linkages  to  the  relevant 
systems  acquisition  efforts  (including  program  office  and  contract  management) — to 
ensure  that  individual  systems  provide  the  operational  capabilities  needed  and  “fit” 
within  the  larger  SoS 

The  execution  of  a  PMO  must  be  consistent  both  with  an  established  vision,  as  well  the 
system  engineering  approach  intended  to  achieve  that  vision. 

These  problems  don’t  result  from  any  lack  of  vision:  vision  statements  abound!  We  have 
observed,  however,  that  there  is  a  lack  of  system  engineering  at  the  SoS  level  (that 
encompasses  the  aforementioned  goals)  to  translate  these  high-level  visions  into  actionable 
plans.  This  suggests  several  possible  courses  of  action: 

•  Articulate  and  assess  the  role  of  a  system  engineering  process  specifically  focused  on 
SoS  acquisition  and  development.  In  particular,  this  process  must  address  the  context  in 
which  the  system  engineering  takes  place,  and  include,  for  example,  funding  and  issues 
of  control.  While  there  has  been  progress  in  the  requirements  perspective  (e.g.,  JCIDS), 
there  seems  to  be  no  corresponding  progress  in  the  construction  (and  lifecycle 
management)  of  the  systems. 
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•  Examine  the  constraints — both  real  and  perceived — to  SoS  acquisition  and  development 
arising  from  the  existing  statutory  and  regulatory  considerations,  and  explore  mitigation 
strategies  that  don’t  require  specific  regulatory  and  legislative  relief. 

•  Assess  the  role  of  current  acquisition  approaches  with  an  approach  that  requires 
interactions  among  processes  to  acquire  an  SoS.  It  is  through  an  assessment  process  that 
one  may  identify  variances  in  the  approaches,  and  begin  to  correct  them. 


In  summary,  there  are  several  issues  arising  from  the  friction  between  the  demands  of 
acquiring  and  fielding  individual  programs  and  doing  so  in  the  context  of  an  SoS.  The 
existing  acquisition  regulatory  and  statutory  framework  (and,  indeed,  the  entire  history  of 
weapon  system  acquisition  and  fielding)  emphasizes  the  individual  system  over  the  SoS. 
Still,  there  is  a  burgeoning  awareness  that  the  key  to  future  success  lies  in  the  identification 
and  adoption  of  effective  practices  for  managing  the  complexities  inherent,  in  a  system-of- 
systems  world. 
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Appendix  A:  Participating  Organizations 


Participants  in  the  October  2004  Future  Force  Workshop  included  representatives  from  the 
following  organizations: 

•  Office  of  the  Assistant  Secretary  of  the  Army  for  Acquisition,  Logistics,  and 
Technology  (ASA/ ALT) 

•  Army  Test  and  Evaluation  Command  (ATEC) 

•  U.S.  Army’s  Communications  -  Electronics  Research,  Development  and  Engineering 
Center  (CERDEC) 

•  Office  of  the  Deputy  Chief  of  Staff  for  Operations  (G-3) 

•  Office  of  the  Deputy  Chief  of  Staff  for  Command,  Control,  Communications,  and 
Computers  (G-6) 

•  Office  of  the  Deputy  Chief  of  Staff  for  Force  Structure,  Resources,  and  Assessment 
(G-8) 

•  Joint  Tactical  Radio  System  (JTRS)  Joint  Program  Office 

•  Program  Executive  Officer  for  Aviation  (PEO  AVN) 

•  Program  Executive  Officer  Simulation,  Training,  and  Instrumentation  (PEO-STRI) 

•  Training  and  Doctrine  Command  (TRADOC)  Program  Integration  Office  (TPIO) 

•  TRADOC 

•  Software  Engineering  Institute  (SEI),  Carnegie  Mellon  University 
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Appendix  B:  List  of  Interoperability  Issues 


General 

•  Lack  of  a  baseline — if  I  don’t  know  what  capability  I  have  today,  how  will  I  know 
what  I  need?  No  place  where  you  can  go 

•  No  consistent  function  def  of  what  systems  are  doing.  What  capability  does  this 
block  have?  This  block? 

•  The  problem  is  we  try  to  hardwire  sys  for  a  force  structure.  But  the  force  structure 

will  change  before  the  system  is  fielded . The  only  way  to  go  is  modularity. 

•  Would  like  an  understanding  what  a  system  of  system  is,  what  it  is  supposed  to  do, 
and  how  well  it  is  supposed  to  do  it.  Would  like  an  ORD  for  system  of  systems. 

•  Given  set  of  systems,  what  capability  does  each  provide  and  what  do  they  need  from 
other  systems.  And  if  appropriate  what  do  they  need  from  a  network?  This  gives  you 
a  way  of  assessing  who  is  doing  what.  Block  3  has  250  tasks — you  can  allocate  those 
tasks  to  the  sys  that  the  arch  performs.  Now  you  have  the  interface  question. . . 

•  C4ISP:  ISP  for  the  future.  How  do  we  all  fit  against  this?  We  don’t  have  this. . . 

•  How  mature  is  the  Army  in  terms  of  its  processes?  Army  needs  an  enterprise  wide 
perspective — portfolio  of  its  projects  (programs) 

•  Wherever  you  say  Army. . .  consider  you  might  say  DoD  and  think  multi  national. 

•  Need  a  methodology  to  support  architecture  development.  They  think  they  have  this 
but  they  are  still  looking  at  a  system  view.  There  is  nothing  to  support  this  from  a 
system  of  systems  view.  Who  owns  this?  Who  can  make  this  happen? 

Programmatic 

•  Requirements  belong  to  the  joint  community.  Then  planning,  programming, 
budgeting  and  execution-all  go  off  to  the  services.  The  problem  is  there  is  no  single 
belly  button.  Designed  to  promote  5  separate  services  and  system  specific 
acquisition. 

•  How  you  buy  a  truck  is  different  from  how  you  deliver  software. 

Operational 

•  TRADOC  family  of  people  responsible  for  defining  the  req.  This  document  justifies 
the  existence  of  every  program. 

•  Organization  structure  impedes  coordination  of  requirements,  later  it  impedes  the 
development  of  the  system  (because  they  follow  the  requirements).  Interoperability  is 
determined  after  the  fact,  band  aided  in. 


34 


CMU/SEI-2005-TN-042 


•  This  is  all  done  at  end  instead  of  at  the  beginning. . ..  What  do  we  want  to  have  on  the 
battlefield  in  2010?  How  are  we  going  to  operate?  What  do  we  think  the  threads  are? 
Build  the  threads  in  conjunction  with  the  warfighter.  Whereas  now,  we  build  systems 
and  then  try  to  glue  things  together  (certification).  (It’s  the  wrong  place  for  systems 
to  come  together  on  the  test  floor.) 

•  If  mission  threads  were  worked  in  advance,  if  TRADOC  got  together  and  went 
through  these  threads. ...  Then  the  acquisition  community  comes  in  with  those 
threads  and  says  this  system  does  this,  this  system  does  this. .  .and  you  give  that  info 
to  the  contactor. . . 

•  There  are  multiple  different  definitions  of  mission  threads.  FCS  doesn’t  have  mission 
threads  it  has  integrated  processes.  Is  there  a  preferred  language  to  use?  Operational 
views  and  system  views.  The  Message  to  Send:  Software  blocking  and  test  floor 
have  found  operational  mission  threads  to  be  useful,  but. . ..  (there  seems  to  be 
limitations  here,  or  this  is  insufficient) 

•  Need  requirements  up  front,  what  systems  are  involved,  then  base  funding  on  this 
prioritization.  It’s  the  30  year  old  problem — that  separates  function  from  the  data. 

•  Definition  and  management  of  the  network  itself  in  the  comms 

•  Can’t  prioritize  functionality  effectively 

•  Now,  things  are  built  on  gaps — on  fixing  holes.  Throwing  money  at  a  app  vs 
throwing  money  at  a  solution 

•  Think  too  stovepiped,  also  solving  today’s  problem  in  today’s  environment 

•  Focused  on  short  term.  Should  Mandate  planning  out  to  5  years. 

•  Rewards  are  wrong 

•  Doctrine  inhibiting  future  force?  Yes,  but  it’s  never  going  to  happen  more  than  3-5 
years  for  now. 

•  Do  we  have  the  right  people?  Do  we  need  more  people?  We  need  as  different  focus. 
Do  people  need  to  stay  in  place  longer?  Yes — 5  years. 

Uncategorized 

•  Lack  of  sharing  of  context  information  at  messages;  better  data  and  user  access. 

•  Lack  of  standards  and  not  implementing  standards  in  a  specified  way 

•  Procurement  versus  development  lifecycle  model  causes  interoperability  problems 
for  when  functions  implemented 

•  Managing  expectations  of  (Government,  contractor,  Congress,  users)  to  limit 
appetite.  Rush  to  add  functions 

•  Not  organized  to  build  a  SOS.  Lack  of  understanding  requirements,  $$$  allocation, 
interaction,  test. 

•  Synchronization  with  joint  systems  with  respect  to  budget,  $ 

•  We  don’t  plan  to  be  interoperable 
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•  Overoptimistic  schedule  crunch 

•  Acquisition  process  is  not  designed  for  a  SOS 

•  Acquisition  process  is  linear  but  you  really  want  evolution  and  there  is  a  mismatch 

•  Development  and  enforcement  of  a  common  data  model(s)  does  not  work 

•  Need  single  management  approach  (overall),  not  FCS,  not  blocking.  Lack  of 
integrated  approach.  Need  more  emphasis  on  blocking. 

•  We  fail  to  communicate  (jargon) 

•  Battlefield  comms  cannot  keep  up  with  user  appetite 

•  We  don’t  hear  each  other 

•  Proliferation  of  requirements 

•  Need  rewards/incentive  for  SOS  at  all 

•  Loss  of  in-house  technical  capabilities  by  Government 

•  Inefficient  application  in  deciding  how  resources  should  be  allocated 

•  Contracting  of  pieces  of  equipment  is  separate  from  blocking  (+USF,  +resetting),  + 
operational  need 

•  Migration  plan  must  be  at  the  appropriate  higher  level,  not  bottom-up  focus.  Network 
not  radios,  fielding 

•  USF  was  fine  until  modularity  and  resetting 

•  Focus  on  material  solutions.  Need  robust  solution-finding  process  that  considers 
DOTMLPF 

•  Need  process  for  measuring  operational  benefit  of  proposed  interoperability 
solutions.  E.g.,  cost/benefits  analysis 
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Appendix  C: 


Acronyms 


ACP 

Army  Campaign  Plan 

AOA 

Analysis  of  Alternatives 

ASA/ALT 

Assistant  Secretary  of  the  Army  For  Acquisition,  Fogistics,  and 
Technology 

ASP 

Acquisition  Support  Program 

ATEC 

Army  Test  and  Evaluation  Command 

AUTL 

Army  Universal  Task  Fist 

BTR 

Below  Threshold  Reprogramming 

C4ISP 

Command,  Control,  Computers,  and  Communications  Integrated 
Support  Plan 

CCA 

Clinger  Cohen  Act  (formerly  the  Information  Technology 
Management  Reform  Act  (ITMRA)  of  1995) 

DOD,  DoD 

Department  of  Defense 

DOTMLPF 

Doctrine,  Organizations,  Training,  Materiel,  header  Development, 
Personnel  and  Facilities 

FAR 

Federal  Acquisition  Regulations 

FCS 

Future  Combat  System 

IRAD 

Independent  Research  and  Development 

ISIS 

Integration  Of  Software  Intensive  Systems 

JCIDS 

Joint  Capability  Integration  and  Development  System 

JTRS 

Joint  Tactical  Radio  System 

ESI 

Fead  System  Integrator 
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ORD 

Operation  Requirements  Document  (replaced  by  the  ICD  -  Initial 
Capabilities  Document) 

PMO 

Program  Management  Office 

SEI 

Software  Engineering  Institute 

SEMP 

Systems  Engineering  Management  Plan 

SETD 

System  Engineering  and  Technical  Direction 

SOS,  SoS 

System  of  Systems 

SOSI 

System-Of-Systems  Interoperability 

SSEI 

System-of-Systems  Engineering  and  Integration 

TRADOC 

Training  And  Doctrine  Command 

USF 

Unit  Set  Fielding 
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